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This case study analyzes five science courses of a United States virtual charter school. Online quizzes and exams 
are provided by the corporate partner, while local teachers have selected report topics, virtual labs and at-home labs 
for students to complete. These assessments were coded for their correlation to the cognitive levels of the revised 
Bloom’s taxonomy and the US NSES (National Science Education Standards). The remembering level was 
associated with the largest number of quiz and exam questions. However, analysis and application were also 
frequently assessed. Most of the standards were assessed at some point. The teacher selected projects address 
science inquiry standards but not additional content standards. The projects often required higher levels of thinking. 
Recommendations for teachers in virtual K-12 (kindergarten through 12th grade) schools are made to select 
additional content that focuses on key concepts and include more application, analysis, evaluation and creation. 
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Introduction 

Online education is a growing trend in K-12 (kindergarten through 12th grade) education. Virtual schools 
offer new modes of instruction to students in the 21st century. These schools deliver complete curricula online 
to students in their homes. In the US, federal law requires publicly funded virtual schools to employ certified 
teachers and administer state assessments for accountability purposes. These state assessments are correlated 
with the state’s content standards. Virtual curricula in the sciences offer positive possibilities, such as diverse 
class offerings for students in remote locations. However, they also lead to questioning how science is being 
taught in these schools. 

The US NSES (National Science Education Standards) affirm “a vision of science education that will 
make scientific literacy for all a reality in the 21st century” (NRC (National Research Council), 1996, p. ix). 
The NSES content standards emphasize scientific reasoning over factual recall. The introduction to the 
standards clearly states that the standards do not exist to prescribe a specific curriculum, but rather they are to 
serve as a guide for various stakeholders to determine whether particular actions will help develop a 
scientifically literate society. The state standards for the virtual school in this study are highly correlated to the 
NSES. 

Bloom’s taxonomy is a tool that can be used as a basis for determining the meaning of particular 
educational goals (Bloom, 1956). The revised taxonomy (Anderson & Krathwohl, 2001) has six levels of 
cognitive processes: remembering, understanding, applying, analyzing, evaluating and creating. They are 
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arranged with the first three being lower level thinking skills and the latter three involving higher order thinking 
skills. Much of the formal assessment in the studied virtual science curriculum involved quizzes and exams 
with multiple choice and free response questions. What levels of knowledge these questions addressed was not 
previously known. This case study analyzed five online science courses for their correlation to the NSES and 
the cognitive levels of the revised taxonomy. 

Bloom’s Taxonomy 

The original Bloom’s taxonomy (Bloom, 1956) was devised as a method of classifying learning objectives 
and assessment of those objectives. Three domains of learning were postulated: the cognitive, affective and 
psychomotor. The cognitive domain related to knowledge and conceptual understanding. The affective domain 
related to attitudes. The psychomotor domain related to coordinated physical abilities and skills. The cognitive 
domain was further divided into six levels that formed a hierarchy from low to high levels of critical thinking: 
knowledge, understanding, application, analysis, synthesis and evaluation. 

The taxonomy was revised by Anderson and Krathworthl (2001). Four knowledge dimensions have now 
been described: factual, conceptual, procedural and metacognitive knowledge. The six categories within the 
cognitive domain have been renamed in verb form and the order has slightly changed. The six levels are now 
remembering, understanding, applying, analyzing, evaluating and creating. 

A critical analysis of the original and revised taxonomies was conducted by Amer (2006). The hierarchy 
found in the original taxonomy was described as being misleading and overly simplistic (Furst, 1994; Anderson 
& Krathwohl, 2001; Kreitzer & Madaus, 1994). For instance, knowledge tasks could be more intellectually 
involved than evaluation tasks. Also, the original taxonomy was grounded in behaviorist philosophy whereas 
educational research supporting constructivism has emerged. 

Amer (2006) argued that the two dimensions of the revised taxonomy (the four knowledge dimensions and 
the six cognitive processes) have several advantages over the original taxonomy. Firstly, they help researchers 
and practitioners to more clearly analyze objectives. Secondly, they help practitioners not to confuse activities 
with objectives. Thirdly, the revised taxonomy clearly links activities with assessment. Finally, it allows for 
more clear alignment of instruction, assessment and objectives. 

The Vision of the US NSES 

Efforts at the reformation of science education in the US have focused upon an emphasis of teaching with 
inquiry methods and focusing on key concepts necessary for scientific literacy (AAAS (American Association 
for the Advancement of Science), 1993; NRC, 1996). The US NSES consist of sets of standards for science 
teaching, professional development, assessment, content, programs and systems. This case study focuses on the 
content standards. Five unifying standards exist for all K-12 science instruction. These standards involve: (1) 
systems, order and organization; (2) evidence, models and explanations; (3) constancy, change and 
measurement; (4) evolution and equilibrium; and (5) form and function. 

The content standards are then divided into three grade level ranges: Kindergarten through the fourth 
grade, the fifth to eighth grade, and the ninth to 12th grade. In each grade level, there are seven content 
standards: (1) science as inquiry; (2) physical science; (3) life science; (4) earth and space science; (5) science 
and technology; (6) science in personal and social perspectives; and (7) history and nature of science. The 
NSES explicitly state that these content standards emphasize the following components less than in traditional 
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classrooms: knowing facts and information, studying subject matter disciplines for their own sake, separating 
science knowledge and process, covering many science topics and implementing inquiry as a set scientific 
method. The NSES have a greater emphasis upon understanding of concepts and inquiry, learning about subject 
matter in context of inquiry, technology, history and the tentative nature of science, and a focusing on a few 
fundamental concepts rather than many disconnected topics (NRC, 1996, p. 113). 

Online Student Assessment 

Distance education has existed since the late 19th century. While it began with correspondence courses 
and eventually involved video delivery, online/ virtual delivery has radically transformed the role of distance 
education in society (Majdalany & Guiney, 1999). A report from the Sloan Corporation (Picciano & Seaman, 
2009) found that 70% of K-12 public school districts had at least one student enrolled in an online course 
during the 2007 to 2008 school year. The National Center for Educational Statistics found that 37% of school 
districts offered online courses in the 2004 to 2005 school year (Zandberg & Lewis, 2008). 

Reasons for the increase in online education has been attributed to increasing the types of courses offered 
to small numbers of students (such as virtual AP (advanced placement) courses for students in rural districts), 
cost effectiveness of delivery and increasing the student to teacher ratio without lowering the quality of 
education. A meta-analysis found that on average, students in virtual education settings performed better than 
those in face-to-face settings. Hybrid or blended learning experiences that utilized both online and face-to-face 
components showed the most performance difference. The report did caution that it should not be interpreted 
that the online medium is superior, but that perhaps the element of extra time allowed online was an important 
factor (USDE (US Department of Education), 2009). 

Online learning may be effective at transmitting explicit knowledge, such as facts, formulas and 
vocabulary. Ozdeimir (2008) argued that online learning is not effective at transmitting tacit knowledge, 
described as “knowing how” rather than “knowing that”. In terms of science education, explicit knowledge 
could include facts about anatomy, atomic structure or classification of landforms. Tacit knowledge could 
include the concepts related to the nature of science, such as understanding that science produces tentative 
knowledge based upon empirical evidence. The higher levels of the revised taxonomy (analysis, evaluation and 
creation) involve tacit knowledge, but the type of questioning will determine whether the student is required to 
question the sources of knowing and inquiry. 

The use of quizzes in virtual education has not correlated to greater student achievement. Maag (2004) 
found no statistically significant difference when comparing a group of nursing students in a math course that 
took online quizzes and one that did not. Stanley (2006) found no statistically significant difference between 
students taking online quizzes and those who only completing homework at university level disease control and 
epidemiology virtual courses. Lewis (2002) compared two groups of university students in an online course. 
One group began a course taking online quizzes and a second group participated in online discussions. After 
half of the course modules were completed, the groups switched their formats. The first group did better than 
the second group when taking online quizzes, but the second group showed no difference when they took 
quizzes and the first group did discussions. Lewis interpreted this result as meaning that the quality of online 
discussions for the first group negated the advantage of taking online quizzes. Tselios, Avouris, 
Dimitracopoulou, and Daskalaki (2001) found that the software platform used to deliver a quiz affected 
university student test performance in their study. The two platforms they compared had similar functionality 
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and content, but the interfaces differed. This study, thus, offers caution against generalizing studies of online 
quizzes. 

One study of the distance learning courses from six colleges (Yang, Hsiao, Liu, & Lin, 2009) found that 
online course objectives emphasized application the most. Remembering was the second most emphasized level 
of the taxonomy. Evaluation was third. Faculty teaching online courses was significantly more satisfied than 
faculty teaching correspondence or television based courses. They found that all types of distance education 
focused on most of the application of knowledge. The application and evaluation components of these courses 
could show some emphasis on tacit knowledge, but it is not described whether the students were required to 
think about the nature of inquiry in the fields of study. 

Science education also involves the use of laboratory experiments to teach about the processes and 
methods of science (NRC, 1996). The national level professional organization in the US for science teachers 
notes that “computers should enhance, but not replace essential ‘hands on’ laboratory activities” (NSTA 
(National Science Teachers Association), 1999). This statement, thus, does not support the role of labs in 
virtual settings. Nevertheless, virtual schools are conducting science courses with simulations. 

Klahr, Triona, and Williams (2007) compared the effects of virtual and physical labs in terms of 
performance and achievement. The seventh and eighth grade students from two private urban middle schools 
crated and tested a series of mousetraps cars. Four groups of students were formed. One condition was whether 
the lab was physical or virtual. A second condition was whether the students were assigned to create six cars or 
they were assigned 20 minutes time to construct as many cars as they could. Students were given a pre- and 
post- assessment. Students were found to be able to construct more cars using the virtual lab. No significant 
effect on understanding was found in terms of whether a physical or virtual lab was used. 

Finkelstein, Adams, Keller, Kohl, Perkins, Podolefsky, Reid, and LeMaster (2005) compared a virtual and 
physical electric circuit lab in a college level introductory physics course. Students who completed the virtual 
lab were able to later build physical circuits faster than those who had only had physical lab experience 
previously. The students who did the virtual lab scored higher on an assessment of conceptual understanding. 
Finkelstein, Adams, Keller, Perkins, Wieman, and the Physics Education Technology Project Team (2006) also 
indentified six key characteristics shown by the PhET (physics education technology) simulations used by his 
group (Retrieved from http://www.phet.colorado.edu) that correlate with quality science instruction. The PhET 
simulations support a provided dynamic feedback, follow a constructivist approach, provide for creativity, 
make explicit microscopic or invisible models, and constrain students “scientific play” constructively. These 
characteristics fit the model of science instruction promoted by the NSES (NRC, 1996). 

Conceptual Framework 

The conceptual framework for this study includes both the NSES and revised Bloom’s taxonomy. The 
content standards of the NSES are used to inform this analysis of the science curriculum of a US virtual charter 
high school. Questions from quizzes and exams were matched with corresponding content standards. The 
revised taxonomy was also linked to each question. This analysis uses the six cognitive processes 
(remembering, understanding, applying, analyzing, evaluating and creating), but it does not include the four 
knowledge dimensions (factual, conceptual, procedural and metacognitive). Because students have access to 
textbooks, the Internet and other resources, while completing the quizzes and exams in these courses, it would 
be difficult to differentiate among the dimensions of knowledge assessed. 
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For instance, a question that would assess conceptual understanding in a proctored classroom environment 
may rather assess factual understanding, because of the access to materials that will turn the exercise into one 
of finding information rather than conceptualizing it. Similarly, a creating question (synthesis under the original 
taxonomy) that may be designed to assess procedural understanding of combining several methods may also 
become a factual question due to the ability of students to find a direct answer to the synthesis from their other 
resources. Therefore, this study does not minimize the importance of this dimension of the revised taxonomy, 
but it would not be adequately addressed due to the mode of delivery of these assessments. 

Research papers, at-home labs (labs physically performed by the student) and virtual labs were also coded, 
but not at the question level, because students had vast amounts of freedom to choose one of the suggested 
projects. This study analyzed the types of content and skills valued by the lab choices and compared this to the 
quizzes and exams. 

Methodology 

This case study analyzed the virtual earth science, physical science, chemistry, physics and biology 
courses used by a virtual charter school in the US. The charter school was authorized by a local school district 
that contracted with a provider of distance education curricula. The school uses state licensed science teachers. 
The provider supplies a curriculum on blackboard course management software that includes roughly weekly 
quizzes, unit exams and a semester exam. Each semester consists of three lessons that are subdivided into 
weekly quiz blocks. 

The local teachers working for this virtual school elected to add application assignments. These 
assignments were in the form of research papers, at-home labs (labs physically performed by the student) and 
virtual labs. Many of these activities redirected students to Websites with simulations or directions for 
conducting the lab at home. Students were given a menu of choices for each of the six lessons in a course. 
There was also a required project for each lesson of each course. These focused on items, such as research 
skills, evaluating sources and using a spreadsheet. The required projects are not included in this study. 

A weekly pace chart is supplied to students telling them which chapters in their physical text book to read 
and which quiz or test to take. Projects and labs are also noted on the pace chart. A lesson viewer is supplied by 
the distance education provider: It is a Web-based version of the text book. Individual teachers have also 
created PowerPoint presentations that teach the content in a different manner. A weekly class discussion/lecture 
is also available via Web conferencing software. 

Each question on the quizzes and exams was coded for the cognitive level of the revised taxonomy that 
corresponded to it. Items coded for remembering focused upon specific pieces of knowledge or facts. Items 
coded for understanding asked students to explain a concept, as multiple choice questions allow students only 
to select answers rather than construct their own answers; they were not coded in the category. Items coded for 
applying had students using content knowledge in a new situation. This typically was the code for problems 
that asked students to identify how a term applied to a given situation rather than just defining it. Items coded 
for analyzing involved questions that had students differentiating among different concepts, and many of these 
questions were matching, since students had to differentiate between terms. Items coded for evaluating required 
students to defend a position. None of the multiple choice questions were coded for evaluating, but several 
constructed response questions had students defending a position. Finally, items coded for creating involved 
students combining several concepts and either describing similarities or developing new knowledge from the 
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given information. This level of thinking was only coded in constructed response questions. 

The NSES content standards consist of five unifying concepts and processes for all grade levels and seven 
standards for grades kindergaren through grade four, grades five to eight and grades nine to 12. Each content 
standard is further broken down into categories of student knowledge or abilities. Then, each of these categories 
is further broken down into fundamental concepts and principles that underlie the standard. For instance, 
content standard A states that as a result of activities in grades nine to 12, all students should develop: (1) 
abilities necessary to do scientific inquiry; and (2) understandings about scientific inquiry. 

Further, under “understandings about scientific inquiry” are six fundamental abilities and concepts. To 
code for not only the overall standard, but also the fundamental concept, principle or ability, the associated 
standard for a question was in the form of A. 2a where “A” refers to content standard A for grades nine to 12, 
the “2” represents the second bullet of standard A as listed in the NSES, and “a” refers to the first of the six 
concepts, principles and abilities. Some of the contents focused on material which fit better with the grades five 
to eight standards. To indicate this, a 5-8 was placed in front of the standard, for instance, 5-8. C. 3a refers to the 
grades five to eight content standard C, bullet 3, first principle/concept. 

The application projects were coded for association with the NSES and revised taxonomy, but only at the 
overall level of the lab. Individual questions were coded for their level on the taxonomy, but since students had 
such a wide choice of activities, the researcher focused on analyzing the overall possibilities that students were 
exposed to. The lab procedure itself was not coded, if it was merely a list of steps for students to follow. Labs 
that required students to develop their own procedure and synthesize their own results were coded with parts of 
standard A that relates to science as inquiry. Labs that required students to physically make a model or 
substance were coded as creating. 

The coding was done by the author of this paper based upon his experience with the standards as a past 
high school science teacher and college methods professor. After an initial coding, the researcher reviewed the 
categories looking for discrepancies and contradictions. Questions using the same verbs were analyzed to 
ensure consistent coding. Finally, member checking was used by sharing the results with the administrator and 
lead science teacher of the school. 


Results 

This section details the analysis of each of the virtual science courses. A review of the coverage of related 
content standards is given. This is followed by a report of the levels of the revised taxonomy used in the 
assessments. A final section reports the results of analyzing the teacher selected application projects. 

Earth Science 

The NSES content standard for earth and space science in grades nine to 12 emphasizes geochemical 
cycles driven by sunlight and the earth’s internal energy, evidence for change and dynamic equilibrium in the 
earth’s systems over geological time, and the scale and nature of cosmic bodies. This is in contrast to the earth 
and space science content standard for grades five to eight which emphasizes an understanding of the earth and 
solar systems in terms of their changes, but not integrating those themes with energy. 

The grades five to eight content standards correlated with 35% of the assessment questions. In fact, the 
three standards correlated with the most questions were all from grades five to eight. The most questions 
addressed the concept of landforms being the result of constructive and destructive forces (5-8.D.lc). The 
assessments had questions about plate movements, earthquakes, volcanism and weathering, but mostly, in 
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terms of defining these terms of identifying examples, such as the cause of the Hawaiian Islands. The 
components of the atmosphere (5-8.D.lh) were assessed by 15 questions, mostly about constituent gases and 
names of layers. The water cycle (5-8. D. If) had 13 questions associated with it. Many questions about ground 
water, cloud types and cloud formation were asked. 

Two of the grades nine to 12 standards had more than 10 questions associated with them. Geologic time 
and the use of radioactive isotope decay in terms of determining age of rocks (D.3b) were related to 11 
questions. The questions related to age of rocks and names of geological eras. The questions did not focus on 
understanding the methodology, but rather upon remembering names of eras as established facts. Ten questions 
addressed global climate and its relation to cloud cover, the earth’s rotation and geological features (D.ld). The 
questions mostly related to air temperature and cloud/front movement. 

The scientific method (unifying standard 2) did receive the attention of 11 questions. However, the 
questions presented the nature of science in a very positivist light rather than emphasizing its tentative, 
revisionary nature. The hypothesis was matched only with the term “educated guess”. A question asked for a 
definition of a control group. This leads to the conclusion that science only studies problems that can be tested 
experimentally. This is problematic in an earth science course, since many theories are developed upon 
observational data. Use of experiments is not a realistic method of studying plate tectonics, climate or the 
history of earth and the universe. 

Three earth and space science standards only had one question associated with them. The first related to 
movement of matter in cycles being driven by the earth’s internal and external energy sources (D.2b). The 
single question was about whether a rise in global temperature would cause sea level rise, so this association is 
not strong. The second relates to the big bang theory (D.4a). A single definitional matching question is asked 
on one quiz. Finally, how stars use nuclear reactions which leads to formation of elements heavier than 
hydrogen and helium (D.4c) had one question that asked about the two most abundant elements in the universe; 
thus, it did not even address theories of the source of all other elements. 

One question asked how old people living in the 19th century thought the earth was. This question is 
probably assuming a North America/European answer. It was classified as G.3d, since it emphasized how 
scientific explanations have changed. One limit is that this belief of the age of the world was not really 
scientific, but it relied on biblical revelation. Another question asked students to identify the steady-state theory. 
This gives historical perspective; however, it could also lead students to think that all theories have opposing 
theories (steady state versus big bang). This complicates discussions of how evolution is not an opposing 
scientific theory of intelligent design; rather, they are different ways of knowing (evolution being based in 
empiricism and intelligent design based in revelation). 

Other standards were completely unaddressed by the questions in the assessments: How atoms flow 
between different reservoirs as parts of geochemical cycles (D.2a); how changes in the physical earth systems 
relate to the evolution of the earth’s features and geological time(D.3c); and theories of formation of galaxies 
and stars (D.4b). Astronomy questions were of a descriptive nature about different phenomena. 

In terms of the revised taxonomy, 61% were remembering, 29% were analyzing, 5% were understanding, 
2% were applying and 1% were evaluating. All of the matching questions were classified as analyzing. The 51 
free response question parts emphasized understanding (35%) and analyzing (20%) the most. The 191 multiple 
choice questions by far emphasized remembering (97%) with the rest categorized as analyzing. Finally, all 24 
true/false questions were classified as remembering questions. The data for earth science are reported in Table 1. 
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Table 1 


Percentage of Earth Science Assessment Questions Classified by the Revised Taxonomy 



Remember 

Understand 

Analyze 

Evaluate 

Create 

Free response 

3.2 

100.0 

9.7 

100.0 

100.0 

Matching 

0.0 

0.0 

84.5 

0.0 

0.0 

Multiple choice 

85.6 

0.0 

5.8 

0.0 

0.0 

True/false 

11.1 

0.0 

0.0 

0.0 

0.0 


Physical Science 

The NSES content standard for physical science in grades nine to 12 emphasizes that students should 
develop understandings between the macroscopic, microscopic and symbolic domains. While grades five to 
eight call for studies in changes of state, solutions and simple chemical reactions, the grades nine to 12 
standards call for connections to be made from physical lab experiences, microscopic knowledge of atoms, ions 
electrons and chemical formulas. The structure of the atom is mentioned as an area where students can learn 
“how scientists know”. 

Nine content standards were assessed by 10 or more questions. The largest numbers of questions (42) were 
related to laws of motion, calculations related to an objects motion and equilibrium (B.4a). Both analyzing and 
remembering classifications were associated with 31% of the questions related to this standard. Standard B.2b 
(24 questions) addresses the nature of elements and the periodic table. Another 24 questions on the assessments 
were about factual knowledge about elements which did not relate to conceptual understanding of the periodic 
law. The chemical reactions standard (B.3c) related to 17 questions, but it classifies reactions as involving 
hydrogen ions (acid/base), electron transfer (oxidation/reduction) and radical reactions. Six other questions 
related to chemical reactions classified reactions as single displacement, double displacement, combustion, 
synthesis or decomposition. The electromagnetic force (B.4e) was correlated with 15 questions mostly related 
to the properties of magnets, but connections were also made to electromagnets, motors and generators. 
Standard B.ld (15 questions) and B.lc (12 questions) both relate to nuclear reactions: the former to radioactive 
decay and the latter to fission and fusion. Chemical bonding (B.2c) was assessed by 13 questions. The 
formation of molecules and association of salts with ionic bonds were made, but the difference between the two 
was not fully analyzed. Standard 5-8. B. 3d relates to circuits (11 questions), and another eight questions were 
asked that involved calculations of voltage, current or resistance in a circuit. Finally, standard B.2a (10 
questions) relates to the properties of electrons in atoms and how they are transferred or shared. Sixty percent 
of these questions used analyzing as students decided what type of ion an atom would form or how many 
covalent bonds could form. 

Four of the grades nine to 12 physical science standards were assessed by only one question: connecting 
the macroscopic states of matter to the microscopic properties of molecules (B.2e), evidence that chemical 
reactions occur in a variety of daily situations (B.3a), the variable reaction rates of chemical reactions (B.3d), 
and the gravitational (B.4b) and electrical (B.4c) forces. No questions compared the strength of the electrical 
force to the gravitational force (B.4d). Also, there was no emphasis on how the electrical force related to most 
observable forces (also B.4d). Fifty-five percent of the questions were not correlated to a national standard. 

In terms of Bloom’s taxonomy, 37% were remembering, 48% were analyzing, 7% were understanding, 8% 
were applying and less than 1% were creating. All of the matching questions were classified as analyzing. The 
142 free response question parts emphasized understanding (33%), analyzing (36%) and applying (25%). The 
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288 multiple choice questions by far emphasized remembering (87%) with the rest categorized as analyzing 
(6%) and applying (7%). Frequency data for physical science are reported in Table 2. 

Table 2 


Percentage of Physical Science Assessment Questions Classified by the Revised Taxonomy 



Remember 

Understand 

Apply 

Analyze 

Create 

Free response 

3.8 

100.0 

62.5 

14.2 

100.0 

Matching 

0.0 

0.0 

0.0 

80.5 

0.0 

Multiple choice 

95.8 

0.0 

37.5 

5.0 

0.0 

True/false 

0.4 

0.0 

0.0 

0.3 

0.0 


Chemistry 

The chemistry course essentially focused on standards B.l, B.2, B.3 and B.5. The structure of atoms (B.l) 
had all parts partially covered. The structure and properties of matter (B.2) also had all parts touched upon. 
States of matter was a common topic for questions, but they did not delve into the molecular nature. The 
chemical reactions standard (B.3) had four of five parts covered in part. No questions addressed the concept of 
a catalyst. Chemical reactions were classified as single displacement, double displacement, combustion, 
synthesis and decomposition rather than in terms of transferring electrons, hydrogen ions or radical reactions. 
Questions about thermal energy were asked (B.5c) and calorimetry problems were given. 

The three standards that had at least 10 questions associated with them were standards B.3c, B.2c and B.lc. 
Chemical reactions (B.3c) were the topic of 32 questions. As with the physical science course, this course 
emphasized identification and classification of chemical reactions rather than connecting the microscopic, 
macroscopic and symbolic domains. Also, as with the physical science course, this course focused on 
classification of chemical reactions by single displacement, double displacement, combustion, synthesis and 
decomposition rather than the emphasis found in the standards. Fourteen questions related to ionic and covalent 
chemical bonding (B.2c), while 13 questions related to fission and fusion (B.lc). Questions also related to the 
binding energy of nuclei and nuclear power plants. 

Three physical science standards only had one question associated with them: the ubiquitous nature of 
chemical reactions (B.3a), the relationship among the macroscopic and microscopic properties of solids, liquids 
and gases (B.2e) and the motion of objects (B.4a). Standard B.4a is better addressed in the physics course. 
Standards B.2e and B.3a were both also found to be minimally covered in the physical science course. 

The majority (64%) of questions were not correlated with one of the NSES. Many of these topics focused 
on calculations mainly relating to gas laws (16 questions), conservation of mass and balancing equations (18 
questions) and stoichiometry (13 questions). Quantum chemistry (13 questions) focused on information about 
electron orbital, which are not included in the national standards. Solutions, mixtures, colloids and suspensions 
(27 questions) were mostly assessed through knowledge about definitions. 

An overall analysis of the curriculum in terms of the revised taxonomy showed that 47% of the question 
were related to remembering, 30% related to applying, 17% related to analyzing, 5% related to understanding 
and less than 1% related to either creating or evaluating. The 55 free response question parts focused most on 
applying (36%), understanding (29%) and analyzing (20%). The matching questions involved both analyzing 
(38%) and applying (62%). Multiple choice questions only related to remembering information 54% of the time, 
while applying (29%) and analyzing (16%) were also utilized. The true/false questions were mostly (92%) 
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remembering information. The frequency data for chemistry are reported in Table 3. 
Table 3 


Percentage of Chemistry Assessment Questions Classified by the Revised Taxonomy 



Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Free response 

2.4 

100.0 

18.7 

18.6 

100.0 

100.0 

Matching 

0.0 

0.0 

7.5 

8.5 

0.0 

0.0 

Multiple choice 

84.5 

0.0 

72.0 

72.9 

0.0 

0.0 

True/false 

13.1 

0.0 

1.9 

0.0 

0.0 

0.0 


Physics 

The physics course focused on the physical science standards B.4, B.5 and B.6. Much of the early course 
focused on motion, forces and calculations associated with those concepts, which explains the reason why 
standard B.4A was associated with 76 questions. Electromagnetism (B.4e) was correlated to 35 questions. Both 
conceptual and analytical questions were asked about kinetic and potential energy (B.5b). Energy transfer from 
hot to cold objects (B.5d) was associated with 21 questions. This standard also refers to entropy, which was not a 
topic in the course. The electrical force (B.4c) was the topic of 17 questions. Standard B.5c (14 questions) is 
associated with the differences between heat and temperature. The conservation of energy (B.5a) was covered by 
1 1 questions. 

Standard B.6 deals with waves and quantized energy. Standard B.6a was the only part not associated with 
any questions, but it was a general standard related to how waves transfer energy through an interaction with 
matter. Twenty-nine questions did assess student knowledge about wave properties and the use of basic wave 
equations, but they did not address this foundational concept. Optics was a topic of 37 questions, but they did 
not relate to the wave properties of light. 

As with chemistry, the majority (51%) of questions were not correlated with a national standard. Other 
than topics already mentioned, the questions asked about the following concepts. In terms of sound, beats, the 
Doppler effects and resonance were assessed. Vectors, units and significant figures were also assessed. 

Large amounts of this course were assessed at the application (28%) and analysis (17%) levels, but 
remembering still was the level of the majority of the assessment questions (51%). Understanding was the focus 
of 3% of the questions. The free response questions emphasized understanding and application equally (36% 
each). However, remembering was involved with 26% of the 50 question parts. All matching questions were 
classified as analysis. The multiple choice questions consisted of 55% remembering, 29% applying and 16% 
analyzing. The 18 true/false questions were 78% remembering. The data for physics are reported in Table 4. 

Table 4 


Percentage of Physics Assessment Questions Classified by the Revised Taxonomy 



Remember 

Understand 

Apply 

Analyze 

Free response 

4.9 

100.0 

12.2 

1.1 

Matching 

0.0 

0.0 

0.0 

18.9 

Multiple choice 

89.8 

0.0 

85.7 

78.9 

True/false 

5.3 

0.0 

2.0 

1.1 


Biology 

The individual standards with the most questions were the content standards for grades five to eight, not 
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the standards for grades nine to 12. Standards 5-8. C. Id (12%), 5-8.C.le (13%) and 5-8. C. 2a (4%) showed the 
highest involvement. One of the course’s six lessons’ quizzes and exams focused on the various animal phyla, 
thus, correlating mostly with standard 5-8. C. Id about specialized cells, tissues and organs cooperating for the 
organism’s functioning. Another lesson focused on human body systems structure and function (5-8.C.le). 
Standard 5-8. C. 2a referred to reproduction in living things and how it can be both asexual and sexual. Of the 24 
questions asked, many related to the forms of asexual reproduction found in several species and to information 
about mating practices of organisms, such as fish. An understanding of meiosis was assessed earlier in the 
course. Standard 8.C.3a noted that organisms need resources to grow, reproduce and maintain stable internal 
conditions. This standard was mostly addressed through an understanding of the difference between 
endothermic and exothermic animals. 

All six grades 9-12 life science standards were addressed at least partially. The six standards address the 
cell, the molecular basis of heredity, biological evolution, the interdependence of organisms, the organization of 
matter and energy in living systems, and the behavior of organisms. 

Nine of the standards were addressed by at 15 questions. In terms of cells, structures and functions (C.la), 
the processes of transcription and translation (C.lc), cell regulation and expression as well differentiation and 
organization (C.ld, C. If) were covered substantially. The role of chromosomes and gamete cells (C.2b) was 
emphasized, especially in regards to the process of meiosis. The role of natural selection and adaptation to 
different niches (C.3a) was covered by different questions. Biological classification was assessed (C.3e) mainly 
in terms of characteristics of different phyla of organisms. Students often had to use analyzing skills to 
differentiate between phyla. The only ecological standard assessed by a large number of questions was the flow 
of energy through ecosystems (C.4b). The NSES choose the nervous system as the body system for high school 
students to analyze at a deep level of understanding. This curriculum spent a lot of time on other human 
systems that are recommended for grades five to eight, so the nervous system did not stand out. Nevertheless, 
several questions on assessments did address the functions at the cellular level of the nervous system (C.6a). 

Items not covered by many questions included genetic regulation of the expression of cellular functions 
(C. If), how cell functions involve chemical reactions (C.lb, C.5a and C.5b) and the role of photosynthesis 
beyond knowledge of reactants, products and conditions (C.le). The concept of descent from a common 
ancestor was not specifically noted in the curriculum (C.3d). Energy loss among trophic levels, cycles and 
environmental limits on species growth were also not addressed. The in depth knowledge about the nervous 
system in terms of evolutionary and human implications were not addressed. 

Table 5 


Percentage of Biology Assessment Questions Classified by the Revised Taxonomy 



Remember 

Understand 

Apply 

Analyze 

Create 

Evaluate 

Free response 

0.6 

100.0 

14.3 

6.1 

100.0 

100.0 

Matching 

6.3 

0.0 

0.0 

53.4 

0.0 

0.0 

Multiple choice 

71.5 

0.0 

71.4 

38.7 

0.0 

0.0 

True/false 

21.6 

0.0 

14.3 

1.8 

0.0 

0.0 


The 58 free response questions mostly assessed understanding (52%), followed up by creating (19%) and 
analyzing (17%). The multiple choice questions mostly assessed remembering (85%) with some analysis (14%) 
and only a few application questions. The true/false questions were almost all remembering (97%). The data for 
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biology are presented in Table 5. 

Teacher Selected Labs and Projects 

The applications projects designated by the local teachers overall did not address content standards 
missing from the quizzes and exams. One notable exception was that chemistry labs did connect to examples of 
chemical reactions in daily life (B.3a), which was not evident in the quizzes and exams. The projects did 
address most of the parts of standard A.l on abilities to do science as inquiry. These skills included students’ 
ability to indentify questions for exploration (A. la), design the experimental procedure (A. lb), use of 
technology and math to investigate (A.lc), formulate and revise scientific explanations (A. Id), analyze 
alternative models (A.le), and communicate and defend conclusions (A. If). 

The number of projects addressing each of these standards is reported by course is reported in Table 6. All 
of the courses offer some labs that require students to design their own procedure, while only a few require 
students to identify and ask their own research questions. Creating explanations and communicating results 
were found in all courses to some extent, but the analysis of alternative explanations was only found in a few 
places. The lead science teacher at the school stated that the labs in middle school (not studied) and physics are 
mostly inquiry. The ones in biology and chemistry are more content and analysis, since proper procedure is 
needed. In all cases, the analysis of the data and reflection back on the hypothesis or purpose is a must. Biology 
and chemistry did show less correlation with the inquiry standard and more correlation with content standards. 

Table 6 


Number of Application Projects per Science as Inquiry Standard 




A.la 

Identify 

questions 

A. lb 

Design 

procedure 

A.lc 

Use math 

A. Id 

Make and revise 
explanations 

A.le 

Analyze 

alternatives 

A. If 

Communicate 

results 

Earth science 

At home 

2 

6 

- 

7 

- 

2 


Virtual 

- 

- 

1 

- 

2 

1 

Physical science 

At home 

3 

8 

1 

10 

- 

5 


Virtual 

2 

6 

4 

6 

1 

4 

Biology 

At home 

1 

2 

1 

3 

- 

1 


Virtual 

- 

- 

- 

- 

- 

- 

Chemistry 

At home 

4 

6 

- 

6 

1 

4 


Virtual 

1 

- 

- 

- 

- 

- 

Physics 

At home 

- 

- 

1 

- 

- 

- 


Virtual 

- 

9 

11 

11 

- 

10 


In terms of correlation between the projects and the revised taxonomy, the projects showed the use of 
higher levels of thinking than found in many of the quizzes and exams. This indicates that the teachers did 
attempt to select projects that would allow students to apply, analyze, evaluate and create. The earth science 
quizzes and exams were heavily focused on remembering while the projects emphasized applying, 
understanding and creating. The physical science projects highly emphasized analysis and creation. Analysis 
had also been emphasized in the quizzes and exams for that course. Biology emphasized analyzing and 
remembering; and remembering had been emphasized in the assessments as well. Chemistry emphasized 
analyzing and applying, whereas many of the assessment questions were about remembering. Physics 
emphasized applying, as did most of the questions on the quizzes and exams. The number of projects at each 
level of the revised taxonomy is reported in Table 7. 
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Table 7 


Number of Application Projects at the Levels of the Revised Taxonomy 




Remember 

Understand 

Apply 

Analyze 

Evaluate 

Create 

Total 

Earth 

At home 

- 

1 

8 

6 

- 

7 

22 

science 

Research paper 

- 

5 

- 

- 

- 

- 

5 


Virtual 

3 

5 

5 

1 

2 

3 

19 


Total 

3 

11 

13 

7 

2 

10 

46 

Physical 

At home 

- 

5 

10 

23 

1 

19 

58 

science 

Research paper 

- 

11 

- 

- 

- 

- 

11 


Virtual 

- 

1 

5 

14 

1 

13 

34 


Total 

- 

17 

15 

37 

2 

32 

103 

Biology 

At home 

3 

3 

3 

9 

- 

8 

26 


Video 

2 

- 

- 

- 

- 

- 

2 


Virtual 

2 

- 

- 

2 

- 

3 

7 


Total 

7 

3 

3 

11 

- 

11 

35 

Chemistry At home 

- 

6 

12 

10 

3 

16 

47 


Research paper 

1 

- 

- 

1 

1 

2 

5 


Virtual 

- 

- 

1 

- 

- 

4 

5 


Worksheet 

- 

3 

1 

4 

- 

2 

10 


Total 

1 

9 

14 

15 

4 

24 

67 

Physics 

At home 

- 

- 

2 

- 

- 

- 

2 


Virtual 

- 

3 

34 

- 

- 

20 

57 


Total 

- 

3 

36 

- 

- 

20 

59 


Discussion 

The teacher selected application projects extend the virtual students’ learning about science as inquiry and 
require them to use higher levels of the revised taxonomy. These projects do not, however, extend the actual 
curriculum itself into the areas of the content standards for earth and space, physical and life sciences. Students 
had control over whether they chose to do at home or virtual labs as well as the level of thinking required for 
the project they chose to complete for each of the six lessons in a course. 

In the future, it is recommended that the virtual school science teachers continue to have the flexibility to 
add projects to standardized quiz and exam assessments. To improve the quality of these projects, the following 
additional recommendations are made: 

(1) To design or choose more projects that require students to ask their own research questions (A. la); 

(2) To incorporate the recognition and analysis of alternative explanations into the procedures and 
expectations of projects (A.le); 

(3) To explicitly introduce students to key concepts of understanding scientific inquiry as found in 
standard A.2. This would involve students learning about the process of science separate of applying it. 
Students could then compare their own methods with those used by practicing scientists; 

(4) To evaluate the method of giving students complete choice of projects to complete. Students could 
continue to pick projects for each lesson that did not challenge them to move beyond the remembering or 
understanding levels; 

(5) Select labs and other projects that address content standards not addressed by quizzes and exams. 
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Examples of teacher selected projects that would address content standards not covered by quizzes and 
exams as follow. One example is given for each course as follows: 

(1) Earth science: Emphasize understanding and analysis of the driving forces behind the flow of matter 
and energy in geochemical cycles using a virtual simulation or student created model; 

(2) Physical science: Emphasize what occurs during reactions more than classifying them. Have students 
describe the relationship between a formula and a video clip, photograph or at-home lab; 

(3) Chemistry: The calculation problems seem appropriate, but include follow up questions that require 
students to connect the results to the macroscopic world in conceptual terms. Have students apply calculations 
about stoichiometry, molarity or gas laws in virtual or at-home labs; 

(4) Physics: Quantitative reasoning is valued in many parts of this course. This value could be maintained, 
but some concepts, such as circular motion, work, capacitors or electric potential could be evaluated at the 
understanding level rather than requiring students to apply formulas to quantities, such as voltage which they 
may not have a meaningful understanding of. 

While knowledge about the human body may be of high interest to students, this course should pick only a 
few body systems to study in depth. Students then focus on understanding how the interconnected parts of the 
system operate rather than remembering part names. They could also use simulations or Internet pictures to 
compare the adaptations of various animals’ anatomy to also connect to the evolution standards. 

Conclusions 

The methodology used in this case study can be applied to other virtual curricula. Since the assessment in 
a virtual course is much more explicit than much of the informal assessment in a face-to-face classroom, these 
courses allow for more direct measures of cognitive thinking levels addressed and the correlation with 
standards. This case study also provides evidence that much of the current assessment found in this school 
emphasizes remembering. As these assessments are “open book” and “open Web”, developers and curriculum 
writers may consider shifting more of the emphasis towards application and analysis. Teachers in virtual 
settings should consider incorporating projects that require Ozdeimir’s (2008) tacit “knowing how” knowledge 
rather than focusing on factual content. In terms of national standard coverage, there were some gaps, but 
overall, the call from the NSES for deeper understanding of a few topics of shallow understanding of many 
topics is of higher importance. Designing a curriculum around fewer concepts may also allow for higher order 
cognitive questions to be asked. While Yang et al. (2009) found application to be emphasized in higher 
education online courses, the courses in this school tended to have large remembering components. Teachers in 
virtual K-12 settings may thus need to add additional material to include other levels of the revised taxonomy. 
As teachers select labs and other projects, they will also need to consider the balance between addressing 
different content standards with the projects than quizzes and exams versus concentrating on depth of 
knowledge over breadth by focusing on the same standards emphasized in quizzes and exams. 
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